The Effects of Placement Examinations and Enforcing Prerequisites on Student Success in Entry-Level Mathematics Courses

Prior to 1995, the placement criteria for entry-level mathematics courses at Texas Tech University were a combination of stipulated high school background requirements and SATM/ACTM score requirements. Advisors were allowed latitude to consider alternate factors for students with marginal scores. As a result, students were admitted into courses where they did not meet the formal prerequisites. There were various consequences of this process which led to dissatisfaction. Two major concerns were: student success rates in mathematics courses, especially in sequential courses such as calculus; and pedagogical issues related to nonhomogenous student populations. In Fall 1996, a mandatory, university-wide placement program was implemented for the entry-level mathematics courses with a requirement that the prerequisites be enforced. There were various immediate impacts on course population distributions with the most obvious being that our remediation population jumped from 629 (Fall 1994) to 1,239 (Fall 1996). We now have data summaries from this first year 1996-97 in which the mandatory placement program was instituted. We also have contrasts with data generated from 1994-95. The data summaries show statistically significant differences for student success rates for various comparable population strata and a potential explanation for those areas in which no difference was detectable. I. Overview of Mathematics Requirements at Texas Tech University and Mathematics Prerequisite Requirements at Texas Tech University Prior to Fall 1995. Texas Tech is a state university with an average enrollment of 24,500 students during fall semesters. The undergraduate population, approximately 19,000 students, is divided among seven colleges, the largest of which are Arts & Sciences, Business Administration and Engineering. The Department of Mathematics and Statistics teaches approximately 7,000 undergraduates each fall semester with the preponderance of those enrollments occurring in service courses for students coming from the colleges of Arts & Sciences, Business Administration and Engineering. The demand for these service courses is driven primarily by two factors: (1) the needs of students in these curricular areas for mathematical preparation prerequisite to their pursuing upper division coursework; (2) the university’s general education requirement of six credit hours of mathematical and/or logical reasoning. Prior to Fall 1995, the formal placement criteria for the entry-level mathematics courses were a combination of stipulated high school background requirements and specified SATM/ACTM scores. Advisors were given, and took, considerable latitude in interpreting these prerequisites. In the department handbook (dated August 1994), the footnote on the page specifying the initial enrollment prerequisites stated, “These are guidelines and not hard and fast rules. The advisor should consider high school GPA as well as other factors for students whose scores are near the cut-off numbers.” The effect of this advisorial latitude on the placement of students was in many ways chaotic. There was (and still is) considerable variability among high schools in the quality of their mathematical preparation of college-bound students. Unless an advisor was familiar with the scholarship level at a particular high school, transcript information about courses taken at that high school and performance in those courses was largely ambiguous. Full-time research faculty serving part-time as advisors within their departments for an appointed period typically did not have instant access to the creditability of a given high school’s scholarship level. Beyond high school transcript information, the only other stipulated requirement was in terms of the student’s SATM/ACTM score. However, SAT and ACT scores are designed to represent a measure of the level of a student’s preparation for collegiate studies. They are generally not designed as predictors for student success in individual courses. The department collected data (SATM/ACTM scores and final course grades) from students enrolled in 1994 in College Algebra (Math 1320) and Calculus I (Math 1351) to conduct a study about how well student success in those courses — passing the course with a “C” or better — could be predicted from SATM/ACTM scores. The results of that study yielded three distinct conclusions: 1) Significant numbers of students were enrolled in Fall 1994 in Math 1320 with SATM scores below 470 (ACTM scores below 18) and in Math 1351 with SATM scores below 580 (ACTM scores below 23) — the stated prerequisite cut-off scores for entering these courses. Advisors were not particularly heeding any of the SATM/ACTM prerequisite criteria. It was not clear whether they were specifically not aware of the prerequisites, whether they were liberally applying latitude in interpreting them or whether they were deliberately ignoring them. 2) In general, SATM and ACTM scores alone were not good predictors for student success or failure in these courses. Had a mandatory prerequisite check been in place to control access to these courses, there was no way to identify a cut-off score for such a check which would minimize the negative impact on students who either: (a) would have passed the course, but would have been denied admission by the cut-off score (it being placed too high); or (b) would have failed the course, but would have been admitted by the cut-off score (it being placed too low). 3) In the extremes (high or low), SATM and ACTM scores could be taken as good predictors of success in these courses. In particular, SATM scores greater than or equal to 610 or ACTM scores greater than or equal to 26 generally could be taken as indicators that the student would successfully complete Math 1320 or Math 1351. Other unmeasurable factors probably influenced advisorial latitude. They include issues such as student or parental pressures to be placed in a given course, say Calculus I (Math 1351), as opposed to a precalculus course for the sake of financial concerns (paying for a nonrequired course), self-esteem (my child already had calculus in high school), etc. Also, included among such factors might be the requirements of a student’s degree plan to judiciously complete certain specified mathematics courses during the freshman year. For a student who is underprepared to begin with Calculus I, the issue arises about whether and which precalculus course the student and advisor should select. The consequences of this placement process were multifold. One obvious area was in the realm of student success rates in their entry-level courses. A survey was taken from the final grade distributions of students in Fall 1994 who were enrolled in sections of Math 1351 taught by faculty members. For this particular selection of sections the success rate was only 43.3%. A critical issue for many of the instructors of those sections was the apparent lack of basic algebra skills by significant segments in their classes. A more general summary of grade distributions for Math 1351 in fall semesters over the time period of 1991-95 showed an overall success rate in Math 1351 of 56.2%. A second impact area, which we will term throughput, relates to the success rates of students who take a sequenced pair of mathematics courses. A survey was taken of the grade distributions for students who enrolled in the sequenced pair Calculus I/Calculus II (Math 1351/Math 1352), taking the first course in the fall and the second course in the spring. The overall throughput rate in the academic years from 1991-92 to 1994-95 for Math 1351/Math 1352 was 34.4%. Another impact area of this placement process was the stratification which was occurring with the nonhomogenous student populations in the individual sections being taught in the department. Distinct bimodal populations emerged in the classroom with significant gaps between their level of preparedness to algebraically cope with the course content, which made it difficult for instructors to responsibly direct the entire class. An additional impact was felt by the instructors of upper division curricular areas in the colleges of Business Administration and Engineering where they needed their students mathematically prepared in order to begin their content areas. They were finding disruption in the delivery of their materials when students were coming into their courses with subpar analytical skills (for which then accommodations had to be made). II. History of the Implementation of a Mandatory Mathematics Placement Examination at Texas Tech University. In the early part of 1995, John Lund, chair of the Mathematics Department at Montana State University, was invited to present a colloquium to the department and to representatives from the colleges of Arts & Sciences, Business Administration and Engineering on the transition which Montana State had gone through in its adoption of a mathematics placement examination as an integral part of its mathematics prerequisite structure. Dr. Lund discussed cycles which a student would go through: taking a course, failing it; repeating the course, failing it; repeating the course again, barely passing it; and then, repeating this cycle in his/her next mathematics course. The problem was the student did not have the appropriate prerequisite skills for the course the first time and that problem was never fixed during any of the attempted repetitions. This cycle was interrupted when deficiencies in mathematical skills were identified and a mandatory correction was enforced to direct the student to an appropriate entry point. That student was then able to proceed linearly from the entry point through his/her mathematics sequences. Dr. Lund’s discussion was well received by the various representatives who attended the colloquium and commitments were made by the colleges of Business Administration and Engineering to support an adoption of a mathematics assessment instrument for the pending summer orientations for Fall 1995. A committee in the department was formed to draft an assessment instrument, which eventually was modeled on the instrument used at Montana State and composed of elements from the Mathematical Association of America (MAA) placement examinations. The colleges of Business Administration and Engineering modified their summer orientation schedules to incorporate the assessment examination and prepared notifications for their prospective participants about the inclusion of a mathematics assessment in the orientation process. The department attended to the details of how to administer the examination, how to process the grading and distribution of results, etc. There were hang ups. Students and their parents did not give due attention to the notification about the inclusion of a mathematics assessment into their summer orientation schedule — some were not aware the assessment was going to be introduced, some were aware, but had made no serious preparations for the assessment, etc. The development and implementation was begun after the deadline for formal changes to the university’s undergraduate catalog — eventually a compromise with students and parents was reached that until the catalog was amended, the assessment could only be interpreted as advisory. Because it was not universitywide, some students and parents took themselves out of the colleges of Business Administration and Engineering over to the college of Arts & Sciences where there was not an assessment process. One of the elements from the MAA placement examinations had an error in its construction. Despite these various problems, the deans in the colleges of Business Administration and Engineering were committed to a mathematics assessment which would appropriately identify deficiencies in their incoming students. Some of the results from the administrations of the Mathematics Placement Examination (MPE) during the summer of 1995 were not promising. At the colloquium, Dr. Lund had discussed that one of the consequences of the implementation of their mandatory assessment program was that in the first year they doubled their remediation population. The instrument we used that first year needed modifications (it was too short in several areas) in order to finely separate students into appropriate cohorts for placement. It needed to be, even if not mandatory in terms of prerequisite placement, university-wide in terms of administration. In late 1995, the department was invited by the university’s Academic Council, comprised of the associate deans from all of the colleges, to present a review of the effects of the MPE which had been implemented in the colleges of Business Administration and Engineering the previous summer. We gathered information about the success rates of students from the Fall 1995 semester (via mid-term grades) and about the difficulties which had been encountered. We hoped that there might be support for a trial university-wide implementation to generate data relative to the validity of the assessment results. However, at the end of the presentation, the dean from Arts & Sciences made a motion that the MPE be adopted for implementation university-wide with enforcement — which was passed. The difficulties of the previous year were largely addressed. The university’s undergraduate catalog was amended to encompass the MPE into the prerequisite structure for the department’s entry-level courses along with a high SATM/ACTM exemption cut-off score. The instrument was lengthened to better address separating students into appropriate cohorts. The university’s summer orientation sessions were modified (increased by a day in length) to accommodate administering, grading and distributing the results of the MPE back to orientation advisors. The most significant impact of the motion which was passed in Academic Council, in terms of labor intensive work, was that the MPE be adopted with enforcement. The university’s current student record system does not track prerequisites for registering students. The department reviews a manual report which is generated for each entry-level section and which flags students who do not meet the prerequisites. Those students have to be individually interviewed to assess their status and to appropriately place them in the correct class or in no class at all. III. Statistical Summary and Comparison of Student Success Rates in Entry-Level Courses Between the Fall Semesters of 1994 and 1996, and Between the Academic Years 1994-95 and 1996-97. Table 1 is a two-way table which identifies the population strata for which success rates were contrasted between preand post-MPE implementations. In Table 2, a statistical comparison of the success rates between various strata in the student populations of several of the entry-level courses for the Fall 1994 and 1996 semesters is drawn. An analysis shows that for the success rates in most of the population strata, there was a statistically significant improvement after the implementation of the mandatory MPE requirement. For example, line c. in Table 2 shows that the analysis identified statistically significant differences at the = 0.002 level in the success rates between the entire student populations for the Fall 1994 and 1996 semesters in three of the four courses which were examined. The exceptions to that trend were in the population strata of incoming students who met the existing prerequisite requirements, which suggests that a lack of adherence in the advising process to the existing prerequisite criteria was a critical issue. In Table 3, a statistical comparison of the throughput rates between various strata in the student populations of the academic years 1994-95 and 1996-97 is drawn. An analysis shows that for the throughput rates in all of the population strata of the calculus sequence, there was a statistically significant improvement after the implementation of the MPE requirement. The adjusted throughput rate considered there factored out individuals who successfully enrolled in the first course, but whose degree requirements did not require them to take the second course. Because at Texas Tech there is a two-semester intermediate sequel between College Algebra (Math 1320) and Calculus I (Math 1351) [namely, Trigonometry (Math 1321) and Analytical Geometry (Math 1350)] the placement examination identifies a cohort who demonstrate proficiency in the second content area, Analytical Geometry, but have deficiencies in the first, Trigonometry. In Table 4, a statistical comparison of the throughput rates between student populations [who proceeded to Math 1351 from a (placed) prerequisite course] in the academic years 1994-95 and 1996-97 is drawn. An analysis shows that for the throughput rates there was a statistically significant improvement after the implementation of the MPE assessment.